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Abstract 

Peer mentoring programs are designed to address problems 
experienced by new students to assist them in making the transition 
to university study. The University of New South Wales offers 
such a program to new undergraduate students. Feedback on the 
program is obtained from participants through questionnaires and 
is used to identify problems and to elicit suggestions for program 
impro vement. The study examines the effectiveness of impro vements 
to the program as a result of evaluation feedback. Findings show 
that improvements to program content and delivery result in better 
outcomes for men tees but that administrative problems still need to 
be addressed. 


P eer mentoring programs are now widely used in universities and colleges 
to assist beginning students to make a successful transition into their 
courses of study. They are conducted at both the undergraduate (Hall, 
2004; Hedges & Mania-Farnell, 2002; Muldoon & Godwin, 2003; Rodger 
& Tremblay, 2002; Stevens & Crase, 2003) and graduate levels (Fugate, 
Jaramillo & Preuhs, 2001, Grant-Vallone & Ensher, 2000; Waldeck et. al., 
1997) as well as for special categories of students, such as those from ethnic 
minority backgrounds (Pope, 2002). 

Many of these programs involve using senior students as mentors. The use 
of peers as mentors rather than faculty or counselling staff aims to provide 
a more supportive environment for addressing transitional problems, such 
as dealing with stress (Jacobi, 1991); accessing services and resources on 
campus; understanding requirements (Rodger & Tremblay, 2002); making 
social contacts (Pope &Van Dyke, 2004); and identifying with the institution 
(Evans & Peel, 1999; Hall, 2004). 

Evidence of the effectiveness of these programs has been documented 
(e.g. Drew et. al., 2000, Hall, 2004), but few studies have sought to identify 
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implementation problems and strategies for improving delivery. In view of 
the widespread use of peer mentoring programs, research is needed on the 
features that make them successful. The purpose of this paper is to identify, 
through evaluation feedback, those components of the program that are 
successful and to ascertain where deficiencies exist so that new components 
can be devised to address these deficiencies. Through this process an 
exemplary peer-mentoring program can be developed that addresses the 
needs of students new to university study and helps reduce the high attrition 
levels among this group. 


Background 

The program at the University of New South Wales has been in operation 
since 2002 and comprises a set of separate faculty or school-based sub- 
programs coordinated by the University Counselling Service. New first year 
undergraduates in selected courses are invited to join a peer-mentoring 
program for the first two to eight weeks (depending on the particular school/ 
faculty) of their first semester at university. Those who accept the invitation 
are assigned to a small group of from four to eight mentees under the 
guidance of a mentor recruited from senior students within the same school 
or faculty. The mentors complete a one-day training course conducted by 
Counsellors from the University Counselling Service. 

Each sub-program is coordinated by a course or school coordinator. In 
2004, there were 27 separate sub-programs across the university; in 2005, 
there were 24 such programs. Further details of the program are described 
in Glaser, Hall and Halperin (2006) and Hall (2004). 

The main aim of this study is to ascertain the extent to which addressing 
problems identified by participants results in improvements in subsequent 
deliveries of the program as perceived by the mentees. 

Gaining feedback from participants on the problems they have experienced 
and on their suggestions for improvement is an integral part of formative 
evaluation (Patton, 1997; Weiss, 1998). This feedback identifies those areas 
of program delivery perceived by the participants to be hindering success, 
and it provides program administrators with a valuable source of information 
from which ideas for program improvement can be generated. 

The source for the problems with the program and recommendations for 
improvement is the feedback obtained from previous years (Hall, 2004). This 
information was gathered in three ways: from questionnaires completed in 
2004 by mentors and mentees, from interviews with a selection of mentees, 
and through meetings with groups of mentors. Problems identified were 
classified into those relating to program organization and administration, 
those relating to content of the program, and those relating to the personal 
qualities of the mentors. 

Organizational and administrative problems included scheduling of 
meetings with mentors, size of mentoring groups, and communication with 
mentees. Areas where improvements had been suggested concerning the 
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content of the program included a greater focus on helping mentees adjust 
to teaching styles, helping them to understand the requirements of the 
university, and including more social functions involving the whole cohort of 
mentees. Problems with the personal qualities of the mentors included their 
lack of knowledge about the university generally; their lack of knowledge of 
the degree programs undertaken by mentees; and, in some cases, a lack of 
friendliness. 

Because the program coordinators have the responsibility for offering 
the program to students within their school or degree course, it is necessary 
to work closely with these coordinators to achieve program improvements. 
There is, however, a large turnover of these coordinators because the role 
is allocated by the school or faculty and is not recognized as any particular 
staff member's responsibility. It is usually allocated to a member of the 
administrative staff rather than the teaching staff. So although feedback on 
evaluation findings is provided to all coordinators, there appears to be little 
attempt at implementing improvements in most of the programs. A survey 
of all coordinators on changes to their mentoring programs in 2005 identified 
only minor changes in all except the Arts and Social Science program. 

The strategy adopted, then, was to work closely with the coordinator of 
the largest peer-mentoring program, namely the Arts and Social Sciences 
program. Through her cooperation, most of the recommendations from the 
2004 evaluation were implemented in the 2005 program. These included 
improved scheduling arrangements for meetings with mentees, briefing 
of mentors about the problems reported by mentees in previous years, 
preparation of additional resource kits to provide mentors with strategies for 
conducting meetings with their mentees, and providing more opportunities 
for mentors to combine with other mentors to organize joint activities and 
social occasions. 

Implementing changes in one program enabled comparison with those 
programs where little or no change had been made. These programs formed 
a de facto control group to determine the effectiveness of the improvements 
made in the Arts and Social Sciences program. This study reports the 
findings from the 2005 Arts and Social Sciences peer-mentoring program in 
comparison with other programs in 2005 and in 2004. 


Method 

Questionnaires were distributed to mentees and mentors in as many of 
the programs as was practicable. Because not all coordinators distributed 
the questionnaires, data was obtained from 12 programs in 2004 and 15 in 
2005. 

The actual method of administration varied from program to program, 
depending on the coordinator. In some programs, emails were sent to both 
mentees and mentors, while in others, random samples of mentees were 
telephoned. The questionnaires were administered in the week following 
completion of the program. 
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The mentee questionnaire in 2004 was completed by 438 mentees 
across 12 of the 27 programs offered that year; in 2005, it was completed 
by 359 mentees across 15 of the 24 program offerings. Response rate varied 
from program to program and averaged 66%. The questionnaire sought 
information from mentees in the following areas: 


1. Whether discontinuation or deferment was being considered 
and, if so, what impact the mentoring program had on their 
decision to stay. 

2. Use made of the mentoring program. 

3. Helpfulness of the mentoring program and the mentor. 

4. Problems encountered with the mentoring program. 

5. Suggestions for improving the program. 


The questionnaire was developed from earlier versions in which open- 
ended questions were included on problems experienced and areas in which 
the program had been helpful (Hall, 2004). Detailed coding of responses to 
these questions identified recurring themes and led to them being rewritten 
as closed questions in the present questionnaire. 


Results 

The focus of the analysis will be on comparison of the Arts and Social 
Sciences Program with the other programs because it is in this program that 
the most extensive modifications were made based on feedback from the 
2004 evaluation. 

Mentees rated the extent to which they made use of the mentoring 
program on a 5-point scale reflecting increasing levels of use. As a first 
test of the effect of improvements to the Arts and Social Sciences program, 
mean ratings for 2005 were compared with those for 2004 in both the Arts 
and Social Science program and in all other programs combined. If program 
improvements in the Arts and Social Sciences program are effective, then 
this should show up in improvements in program use. That is, mentees who 
believe that the program is providing them with the assistance they need 
should make greater use of it than those who do not. Figure 1 shows the 
mean rating of program use by mentees in 2004 and 2005. Mentees rated 
their use of the program on a five-point scale from little or no use (1) to 
extensive use (5). 
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Figure 1: Mean ratings of use of peer mentoring program for 2004 and 
2005. Ratings were on a five-point scale from 1 (little or no use) to 5 
(extensive use). 


A two-way analysis of variance with program (Arts vs. others) and Year 
(2004 vs. 2005) as the main effects found that use of the Arts and Social 
Science program by mentees showed a much greater increase than that of 
all other programs combined (FI, 738 = 9.59, p = .002 for the interaction 
term). Overall use increased from 2004 to 2005 (FI, 738 = 31.93, p = .000) 
while overall program main effects showed no significant differences. 

Mentees also rated helpfulness of the program overall and in the main 
areas covered by the program, as well as helpfulness of their mentor. 
The ratings were made on a five-point scale reflecting increasing levels of 
helpfulness. 

The percentage of mentees rating the program helpful for each of these 
areas is shown in Table 1 together with results of the significance tests for 
the overall change from 2004 to 2005 and the change for the arts program 
and other programs separately. 

Table 1 shows that for every measure of helpfulness, there is an 
improvement from 2004 to 2005 for the arts program, and in every case, this 
improvement is greater than that for the other programs. The improvement 
for the arts program is statistically significant at the .05 level in four of the 
eight areas and in none of them for the other programs. These areas are as 
follows: help navigating around the university, help with teaching style, help 
with social contacts, and overall help from the mentoring program. 
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Table 1 

Percentage of mentees rating the mentoring program helpful overall and in 
each of the various areas addressed by the program. 



Program 

Significance of Effects 8 


Arts 

Other 




Area of Helpfulness 

2004 

2005 

2004 

2005 

Year 

Arts 

Other 

Overall ratings of helpfulness: 

Helpfulness of the mentoring 
program overall 

38.9 

62.7 

45.5 

47.7 

.070 

.013 

.664 

Helpfulness of the mentor 

54.3 

70.5 

55.3 

60.4 

.115 

.213 

.523 

Ratings of helpfulness in specific areas: 

Finding my way around 
the university 

38.9 

48.5 

26.4 

30.7 

.003 

.017 

.281 

Help with teaching style 

54.3 

26.9 

12.3 

18.1 

.015 

.010 

.487 

Help with social contacts 

21.6 

42.0 

31.4 

30.9 

.340 

.047 

.834 

Help access university services 

29.7 

43.3 

18.7 

29.9 

.000 

.317 

.050 

Help feel part of 
university community 

32.4 

42.1 

30.3 

30.2 

.629 

.165 

.915 

Help in understanding 
university requirements 

35.1 

47.5 

30.0 

35.7 

.106 

.095 

.663 


Significance levels for chi-square tests of the relationship between rated helpfulness and year 
overall (first column), for Arts mentees (second column) and other mentees (third column). 


Mentees were also asked to identify the problems they experienced with 
the program by responding to questions about the problems most often 
encountered, such as communications with mentor, scheduling clashes, and 
issues being neglected. Table 2 shows the percentage of mentees reporting 
which of these problems they had experienced with the mentoring program 
along with the results of the chi-square significance tests of the change in 
these responses overall and in the arts and other programs separately. 

Table 2 shows that mentees reported a reduction in three of the four 
problem areas in the arts program and all four in the other programs, 
although this reduction was significant in only two of the areas. Only one of 
these areas— frequency of communications with mentor— was the reduction 
significant for the arts program and not for the other programs. 

Another measure of overall effectiveness of the mentoring program is 
provided by the responses to the question concerning the impact of the 
program on the decision by students considering discontinuing or deferring 
their study at the university. Table 3 shows the percentage of students 
rating this impact as considerable (4 or 5 on a five-point scale) for arts and 
other mentees for each of the years 2004 and 2005. Also shown are the 
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significance levels for chi-square tests of the relationships between year 
and ratings separately for arts and other programs and the contingency 
coefficient as a measure of the strength of the relationship. 


Table 2 

Percentage of mentees responding yes to questions concerning problems 
they experienced with the mentoring program. 


Area of Helpfulness 

Program 1 

Significance of Effects 3 [ 

1 Arts 1 

1 Other [ 

Year 

Arts 

Other 

2004 

2005 

2004 

2005 

Finding my way around 
the university 

41.9 

31.3 

22.4 

14.1 

.062 

.212 

.021 

Help in understanding 
university requirements 

25.8 

10.7 

21.9 

17.0 

.011 

.017 

.196 

Help feel part of 

27.4 

8.0 

15.2 

8.3 

.001 

.001 

.023 

Help in understanding 
university requirements 

12.9 

15.2 

20.8 

17.0 

.283 

.854 

.317 


Significance levels for tests of difference between proportion of mentees endorsing each statement in 2004 
compared to 2005 overall (first column), for Arts mentees (second column), and other mentees (third column). 


Table 3 

Percentage of mentees reporting that the mentoring program made a great 
deal of impact on their decision to stay at university. 


Program 



Significance of Effects a 


Contingency Coefficient b 

Arts 


Other 







2004 

2005 

2004 

2005 

Year 

Arts 

Other 

Arts 

Other 

4.5 

11.1 

3.3 

2.9 

.002 

.016 

.056 

.445 

.261 


Significance level for chi-square test of the relationship between ratings of 
impact and year for all mentees and separately for arts and others. 

^Measure of the strength of the relationship between rated impact and year. 


Although the percentages in Table 3 show that the mentoring program 
had a great deal of impact on mentees' decision to stay at university, the 
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increase in this percentage for the arts program is quite large and significant. 
The program need only be responsible for one student deciding to continue 
at university for it to pay for itself many times over. 

Analysis of the mentee's responses in 2005 to the open-ended question 
soliciting comments and suggestions on the program revealed ongoing 
administrative problems with scheduling, attendance at meetings, and 
communication with mentors in both the arts and other programs. Few 
comments were made on the actual content of the programs. Those that 
were made focused on the need to prepare students for various styles of 
teaching at university. This was the lowest rated area of helpfulness of the 
program (see Table 1). One Arts and Social Science mentee commented, 
"I think preparing students to handle the workload and university styles of 
learning should be a major focus of the mentoring program." However, the 
overwhelming majority of comments were favourable about the program, 
the mentor, or both. For instance, the following arts mentee comment 
represents the majority of comments: "I thought the program was truly 
amazing! I loved my mentor. He became a good friend of the group and 
helped us In a time of need. I definitely feel peer-mentoring needs to be 
continued for many years to come." 

Discussion 

The focus of this study was on the impact of improvements to the delivery 
of peer mentoring programs for first year undergraduate students. Feedback 
from evaluation of the 2004 programs was provided to coordinators of the 
2005 programs, but substantial changes were implemented in only one of 
those programs, namely the Arts and Social Science program. 

These changes focused on both administrative and content areas of the 
program. The key administrative problems were scheduling of meetings and 
communications with mentors. Mentees had commented that meetings with 
their mentor(s) often clashed with their classes and that in some instances 
mentors had not contacted them about the program. These problems were 
addressed by (a) providing all mentees details of meeting times of mentoring 
groups and giving them the opportunity to change to an alternative group at 
a more convenient meeting time, and (b) providing mentors with an office 
phone to enable them to contact their mentees. 

Content problems that had been identified from previous studies included 
lack of attention to university requirements and teaching styles as well as 
insufficient opportunities to meet other students in a social context. These 
problems were addressed by providing additional training sessions for 
mentors in the Arts and Social Science program. These sessions explored 
strategies for helping mentees adjust to university study, including some 
strategies on how to help mentees cope with differences between university 
and high school. There were also suggestions for mentors on possible ways 
to organize social occasions in combination with other mentoring groups to 
provide greater opportunities for social contacts with mentees. 

These training sessions were in addition to those that had already 
been undertaken by mentors in all programs. This university-wide training 
program, provided by the University Counselling Service, focused on 
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responsibilities of the mentor that included ethical issues but dealt minimally 
with program content. No other program coordinators provided additional 
training sessions for their mentors. 

Implications 

The findings from this study show that adding the extra training for 
mentors in the Arts and Social Science program and improving administrative 
arrangements result in better ratings of the program by mentees in 
comparison with mentees in other programs. Arts and Social Science 
mentees also reported experiencing fewer problems with the program and 
were more likely to report that the program helped them in their decision to 
continue at university rather than to discontinue or defer. 

While administrative arrangements such as communication, scheduling 
and meeting facilities are important preconditions for the success of a peer- 
mentoring program, it is the program content that will be the determining 
factor in program success. A well-organized program that does not meet the 
needs of mentees is not going to produce helpful outcomes for them. 

Further Research 

The present study has sought to identify some of these needs and devise 
strategies in the mentoring program for addressing them. The strategies 
that have contributed most to the improved outcomes (as shown in Table 1) 
include helping mentees find their way around the university campus, adjust 
to teaching styles, and make social contacts. These content areas seem to 
be important ingredients in a successful peer-mentoring program. There 
is, however, further room for improvements as helpfulness ratings in most 
areas are still below 50%. 

Above all, having a sympathetic and friendly mentor appears to be a 
crucial ingredient in the success of the program, as many mentees have 
commented as is evidenced by the high rating of helpfulness of the mentor 
shown in Table 1. 

Although program revisions have addressed many of the problem 
areas, there still remain problems of clashes of mentoring meetings with 
classes and communication with mentees as shown in Table 2. Further 
improvements will need to address these problems as well as the need to 
find more innovative ways to deliver the program content. One strategy 
being considered is to set up a web site for the mentoring program to enable 
mentors and mentees to communicate on-line. This will enable mentees not 
able to attend meetings to post their questions and have them answered by 
their mentors on-line. 


Conclusion 

If peer-mentoring programs are to be expanded in universities 
and colleges, detailed studies need to be conducted to determine the 
effectiveness of these programs and to identify those features contributing 
to their success. This study has made a start in this process by showing 
that including guidance on approaching university study, help on locating 
campus resources and facilitating social contacts with other students, and 
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improvement of the mentoring experience of first year students. Further 
research should focus on identifying further areas helpful to mentees and on 
the personal qualities of mentors that contribute to a successful mentoring 
outcome. 
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